Method and apparatus for human-computer interaction in display device, and computer device and storage medium

ABSTRACT

The present application provides a method and apparatus for human-computer interaction in a display device, and a computer device and a storage medium. The method comprises: a display device acquiring current image data of a user, and displaying multiple pieces of different candidate data in different display regions when it is detected that the user is in a squarely viewing state, so that the user browses the candidate data; and the display device identifying a target display region which is focused on, and reading candidate data corresponding to the target display region, and executing an operation corresponding to the read candidate data.

CROSS REFERENCES TO RELATED APPLICATION

This patent application claims priority to Chinese patent applicationNo. 2018105505301, filed on May 31, 2018, entitled “METHOD AND APPARATUSFOR HUMAN-COMPUTER INTERACTION IN DISPLAY DEVICE, AND COMPUTER DEVICEAND STORAGE MEDIUM” the disclosure of which is hereby incorporated byreference in its entirety.

TECHNICAL FIELD

The present disclosure relates to a method, a device, a computer device,and a storage medium for a human-computer interaction in a displaydevice.

BACKGROUND

Human-computer interaction refers to a process of information exchangebetween human and a device, in which a defined task is completed byusing a certain dialogue language between human and the device in acertain interactive manner.

With the development of science and technology, application fields ofhuman-computer interaction technology, for example, as small as a playbutton of a radio, as large as a dashboard on an airplane or a controlroom of a power plant, are becoming wider and wider. Users cancommunicate with a system and perform operations by using ahuman-computer interaction interface. At present, in the human-computerinteraction technology, a human-computer interaction mode is single,which requires the user to issue a specific voice command, and then inresponding to this, a display device perform further operations.

SUMMARY

Accordingly, as for above problems, it is necessary to provide a method,a device, a computer device and a storage medium for a human-computerinteraction in a display device.

A method for a human-computer interaction in a display device includes:

obtaining current image data of a user;

when detecting that the user is in a front-view state, displayingmultiple different candidate data in different display areas;

recognizing a target display area where the user is gazing;

reading the candidate data corresponding to the target display area, andperforming an operation corresponding to the read candidate data.

In one of the embodiments, the method further includes, after readingthe candidate data corresponding to the target display area, andperforming the operation corresponding to the read candidate data:

when detecting that the user is not in the front-view state,interrupting the operation corresponding to the read candidate data.

In one of the embodiments, the when detecting that the user is in thefront-view state, displaying the multiple different candidate data inthe different display areas includes:

when detecting that the user is in the front-view state, counting timefor which the user is in the front-view state;

when the time for which the user is in the front-view state exceeds apreset time threshold, displaying the multiple different candidate datain the different display areas.

A device for a human-computer interaction in a display device includes:

an image data acquisition module configured to acquire current imagedata of a user;

a display module configured to display multiple different candidate datain different display areas when detecting that the user is in afront-view state;

a recognition module configured to identify a target display area wherethe user is gazing; and

an execution module configured to read candidate data corresponding tothe target display area and perform an operation corresponding to theread candidate data.

A computer device includes a processor storing a computer program and amemory. When executing the computer program, the processor implementsthe steps of the method as described above.

A computer-readable storage medium stores a computer program thereon.When being executed by a processor, the computer program implements thesteps of the method as described above.

The details of one or more implementations of the subject matterdescribed in this specification are set forth in the accompanyingdrawings and the description below. Other potential features, aspects,and advantages of the subject matter will become apparent from thedescription, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The following drawings of the present disclosure are used herein as partof the present disclosure to understand the present disclosure.Embodiments of the present disclosure and description thereof areillustrated in the accompanying drawings to explain the principle of thepresent disclosure.

FIG. 1 is an application environment diagram of a method for ahuman-computer interaction in a display device according to anembodiment.

FIG. 2 is a schematic flowchart of the method for the human-computerinteraction in the display device according to an embodiment.

FIG. 3 is an application environment diagram of a method for ahuman-computer interaction in a display device according to anotherembodiment.

FIG. 4 is a schematic flowchart of the method for the human-computerinteraction in the display device according to another embodiment.

FIG. 5 is an application environment diagram of the method for thehuman-computer interaction in the display device according to anotherembodiment.

FIG. 6 is an application environment diagram of the method for thehuman-computer interaction in the display device according to anotherembodiment.

FIG. 7 is an application environment diagram of the method for thehuman-computer interaction in the display device according to anotherembodiment.

FIG. 8 is an application environment diagram of the method for thehuman-computer interaction in the display device according to anotherembodiment.

FIG. 9 is a structural block diagram of a device for a human-computerinteraction in a display device according to an embodiment.

FIG. 10 is a view of an inner structure of a computer device accordingto an embodiment.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Although the conventional human-computer interaction mode can realizethe human-computer interaction function to a certain extent, due to thesingle interaction mode, certain specific pose actions need to bepre-defined. The interaction process is not very natural, and to acertain extent, which brings inconvenience to the user when performingoperations.

In order to illustrate objects, technical solutions and advantages ofthe present application clearer, the present application will be furtherdescribed in detail below with reference to the accompanying drawingsand embodiments. It should be understood that the specific embodimentsdescribed here are only used to explain the present application, and notto limit the present application.

Methods for a human-computer interaction in a display device provided inthis application can be applied to an application environment as shownin FIG. 1. A user appears within a visual angle range of a displaydevice. The display device (television) obtains current image data ofthe user. When detecting that the user is in a front-view state,multiple different candidate data are displayed in different displayareas. The user browses the candidate data in the display device. Thedisplay device recognizes a target display area where the user isgazing. The display device reads the candidate data corresponding to thetarget display area, and performs an operation corresponding to the readcandidate data. The display device may specifically be a television, atablet computer, a tiled display device, a vehicle-mounted displaydevice, or the like.

In an embodiment, as shown in FIG. 2, a method for the human-computerinteraction in a display device is provided. Taking the method appliedto a display device as an example for description, the method includesthe following steps:

S200: obtaining current image data of a user.

The display device may obtain the current image data of the user bymeans of a camera built-in or external to the display device.Specifically, the camera can be provided on the display device. Thecamera can be rotated by 360 degrees to capture images of the user andobtain the current image data of the user. Furthermore, the camera canbe provided directly in front of the display device. The camera acquiresthe current image data of the user who enters the visual angle range ofthe display device.

S400: when detecting that the user is in a front-view state, displayingmultiple different candidate data in different display areas.

A front-view recognition is performed on the current image data of theuser obtained in step S200. When detecting that the user is in thefront-view state, the multiple different candidate data are displayed inthe different display areas. For simplicity, “front-view recognition”here can be understood as a “switching” action. When detecting that theuser is in the front-view state, the display device immediately displaysthe multiple different candidate data in the different display areas.Furthermore, the detection can be performed by comparing the currentimage data of the user with historically obtained image data of the userin the front-view state. In practical applications, front-view imagedata of the display device in the front-view state acquired by an imageacquisition device is first obtained. The front-view image data of thedisplay device in the front-view state can be the data acquired in ahistorical record or the data acquired on the spot. The imageacquisition device can be a camera or the like. Here, the front-viewimage data of the display device in the front-view state is acquired bythe image acquisition device. The image acquisition device can beprovided on the display device, or on an auxiliary device or aperipheral device of the display device. For example, when the device isa television (TV), the image acquisition device can be provided on theTV or on a set-top box that is matched with the TV. More specifically,after performing image processing and image target coordinate conversionon the front-view image data of the display device in the front-viewstate captured by the camera, a relative position of the device and theuser's face can be determined. That is, face image data of the user inthe front-view state of the display device can be obtained. Thedetermination that the display device is in the front-view state can beachieved by using techniques such as head pose estimation or gazetracking.

S600: recognizing a target display area where the user is gazing.

Adopting techniques such as head post estimation and gaze tracking canaccurately identify the target display area where the user is gazing.Specifically, different data is displayed in the different display areason the display device. The user directly gazes (watches) the targetdisplay area according to his own needs when browsing the theses data.In practical applications, the position where the user is gazing can berecognized based on the gaze tracking technology. Staying time of theuser's gaze in each display area is cumulatively counted. The candidatedisplay area corresponding to the staying time that firstly andcumulatively exceeds a preset gazing time threshold is selected as thetarget display area where the user is gazing. For example, as shown inFIG. 3, currently, three display areas on left, middle, and right of theTV show contents of “call the granddaughter, the eldest son, and theyounger son”. The user has been gazing the middle display area for thelongest time. That is, the target display area where the user is gazingis recognized as the middle display area.

S800: reading the candidate data corresponding to the target displayarea, and performing an operation corresponding to the read candidatedata.

The relationship between each display area and the corresponding data isstored in the display device. After the target display area isdetermined in step S600, the display device directly reads the storedcandidate data corresponding to the target display area, runs thecandidate data, and performs corresponding operations. Based on takingthe embodiment shown in FIG. 3 as an example, after determining that themiddle display area is the target display area in step S600, the displaydevice reads the data of “call to an eldest son” and performs theoperation of“call to eldest son”. Optionally, when the display devicedoes not have a communication function, a peripheral device (acommunication device) can be controlled to perform the operationcorresponding to the read candidate data.

In the above-mentioned method for the human-computer interaction in thedisplay device, the display device obtains the current image data of theuser. When detecting that the user is in the front-view state, themultiple different candidate data are displayed in different displayareas. The user browses the candidate data. The display devicerecognizes the target display area that is being gazed, reads thecandidate data corresponding to the target display area, and performsthe operation corresponding to the read candidate data. In the wholeprocess, user's needs can be perceived without excessive operation,which realizes a convenient human-computer interaction, and bringsconvenient operation to users.

As shown in FIG. 4, in one of the embodiments, after step S800, themethod further includes:

S900: when detecting that the user is not in the front-view state,interrupting the operation corresponding to the read candidate data.

When the display device detects that the user is not in the front-viewstate, the operation corresponding to the read candidate data isinterrupted. Continuing to take the embodiment shown in FIG. 3 as anexample, during that the TV is performing the operation of calling tothe eldest son, when detecting that the user is not in the front-viewstate, the operation is interrupted. Optionally, step S900 may beperformed within a limited time range. That is, when detecting that theuser is not in the front-view state within a limited time range (forexample, 10 seconds), the operation corresponding to the read candidatedata is interrupted, so the user does not have to maintain thefront-view state for a long time, which brings convenience to users.When the user wants to end the current operation, the user can be in a“non-front-view state” with the display device. When detecting that theuser is not in the front-view state, the TV will interrupt the currentoperation. For example, the user currently wants to end the call withthe eldest son, and the user will turn his head away, such that the TVrecognizes that the user is not in the front-view state, and interruptsthe call with the elder son.

As shown in FIG. 4, in one of the embodiments, S400 includes:

S420: when detecting that the user is in the front-view state, countingtime for which the user is in the front-view state;

S440: when the time for which the user is in the front-view stateexceeds a preset time threshold, displaying the multiple differentcandidate data in the different display areas.

Here, as for the front-view state, the requirement for the preset timethreshold is further introduced. Only when the user is in the front-viewstate for more than the preset time threshold, the multiple differentcandidate data will be displayed in the different display areas. Theuser gazes at the TV (in the front-view state) for a certain time (3seconds), and the TV displays the different candidate data in thedifferent display areas. By adopting the above-mentioned method,misoperation of the user during daily use can be avoided, bringingconvenience to the user.

In one of the embodiments, when detecting that the user is in thefront-view state, the displaying the multiple different candidate datain the different display areas includes: according to the current imagedata of the user, adopting a front-view recognition technology topreliminarily detect whether the user is in the front-view state; whenthe preliminary detection result is that the user is in the front-viewstate, acquiring the face image of the user; according to the face imageof the user, performing recognition processing; according to therecognition processing result, displaying the multiple differentcandidate data in the different display areas. The recognitionprocessing includes at least one of a face recognition, an agerecognition, an facial expression recognition, and a gender recognition.

In this embodiment, the front-view recognition, combined with the facerecognition, or the age recognition, or the facial expressionrecognition, or the gender recognition is adopted to more accuratelydisplay the interaction data of the user. Specifically, firstlyaccording to the current image data of the user, the front-viewrecognition technology is adopted to preliminarily determine whether theuser is in the front-view state. When the user is in the front-viewstate, the face image of the user is acquired. At least one processingof the face recognition, the age recognition, the facial expressionrecognition, and the gender recognition is performed on the acquiredfrontal face image. According to the processing result, the multipledifferent candidate data are displayed in the different display areas.For example, when based on the front-view recognition combined with theface recognition, the user's identity is recognized, and the candidatedata associated with the user's identity, such as the user's addressbook, and avatars of the user's relatives, can be displayed in differentdisplay areas. When based on the front-view recognition, combined withthe face recognition and the facial expression recognition, the user'sidentity is recognized, and it is preliminarily recognized that the userhas a happy facial expression at present, the candidate data associatedwith the identity and in which the user is interested, such as user'sfavorite TV programs, user's favorite pictures, photos, and the like,can be displayed in the different display areas.

In one of the embodiments, the above-mentioned method for thehuman-computer interaction in the display device further includes: whena size of the face image of the user is greater than a preset face imagesize threshold, displaying the multiple different candidate data in thedifferent display areas.

As a distance between the user and the display device is different, thesize of the face image of the user in the user's images acquired by thedisplay device is different. In addition, the human-computer interactionis effective only when the user is within a certain distance from thedisplay device. For example, during that the user performs thehuman-computer interaction with the TV, if the user is tens of metersaway from the TV, the human-computer interaction between the user andthe TV is obviously an invalid human-computer interaction action. Inthis embodiment, based on the above-mentioned principles and actualapplication requirements, only when the size of the user's face image isgreater than the preset face image size threshold, the multipledifferent candidate data can be displayed in the different displayareas. The preset face image size threshold can be a face image acquiredwhen presetting the distance between the user and the display device,for example, an acquired face image pre-acquired when the user is 3meters away from the TV device, the size of corresponding frontal faceimage is obtained as the preset the face image size threshold.

In one of the embodiments, the when the preliminary detection result isthat the user is in the front-view state, acquiring the face image ofthe user includes: when the preliminary detection result is that theuser is in the front-view state, performing the face recognition on theuser; and when recognizing the user as a known legitimate user,acquiring the face image of the user.

When detecting that the user is in the front-view state, the facerecognition is performed on the user to determine whether the user is aknown legitimate user. When the user is a known legitimate user, thefrontal face image of the user is acquired. In this embodiment, a dualmechanism combining the front-view recognition and the user identityrecognition is adopted to identify whether multiple different candidatedata needs to be displayed in different display areas, so as to avoidmisoperation and bring convenience to the user.

In one of the embodiments, the above-mentioned method for thehuman-computer interaction in the display device further includes:

When recognizing the user's identity by the face recognition, displayingmultiple different candidate data associated with the user's identity inthe different display areas.

The candidate data is associated with the user's identity. For example,when recognizing the user's identity, multiple contact informationassociated with the user can be displayed in the different displayareas. For example, the contact information of the daughter, the son,and the mother can be displayed in the three display areas on the left,middle, and right.

In one of the embodiments, the recognizing the target display area wherethe user is gazing includes: adopting a head pose recognition technologyto confirm an orientation and an angle of the user's head; according tothe orientation and the angle of the user's head, recognizing a displayarea being browsed by the user, and using the display area being browsedby the user as the candidate display area; respectively counting thestaying time of the user gaze in the different candidate display areas;selecting the candidate display area with the longest staying time asthe target display area where the user is gazing.

It should be understood that, although the various steps in theflowcharts of FIGS. 2 and 4 are displayed in sequence as indicated byarrows, these steps are not necessarily performed sequentially in theorder indicated by the arrows. Unless clearly illustrated herein,performing these steps is not strictly restricted in order, and thesesteps can be performed in other orders. Moreover, at least part of thesteps in FIGS. 2 and 4 may include multiple sub-steps or multiplestages. These sub-steps or stages are not necessarily performed at thesame time, but can be performed at different times. These sub-steps orstages are not necessarily performed in sequence, but can be performedin turn with or alternately with at least part of other steps, orsub-steps or stages of the other steps.

In order to further explain the technical solutions and effects of themethod for the human-computer interaction in the display device of thepresent application in detail, the detailed description will be madebelow by taking the application to the TV as a specific applicationexample, in conjunction with FIGS. 3, 5, 6, 7 and 8. When applied to theTV, the method for the human-computer interaction in the display deviceof this application includes the following steps:

1. As shown in FIG. 5, when the user faces the TV screen, the TVacquires the image data of the user.

2. The TV determines whether the user is in the front-view stateaccording to the acquired image data of the user, the details are asshown in FIG. 5 and FIG. 3. The user is frontally facing the center ofthe TV. The TV detects that the user is in the front-view state, andmultiple different candidate data are displayed in different displayareas on the TV. Specifically, three contacts of the granddaughter, theeldest son, and the younger son are displayed on the left, middle andright of the TV, and voiceover or explanatory data is displayed in otherblank areas (this part of the content may not be implemented).

3. As shown in FIG. 6, the user slightly swings his/her head left andright to select one of the contacts by gazing at it. A video call can bemade after staying there for 3 seconds. The TV adopts a dual recognitionmechanism combining the front-view recognition and the user's identityrecognition to identify that the current user is a legitimate user, andthat the target display area where the user is gazing is the left area.

4. The TV reads the candidate data corresponding to the left area andperforms the corresponding operation. As shown in FIG. 7, the TV readsthe data related to the granddaughter's video call and starts a videocall with the granddaughter.

5. As shown in FIG. 8, when the video call is connected, the image ofthe video call is displayed on the TV. When the user needs to interruptor end the video call, the user can turn his/her head and gazing at theTV sideways. The TV detects that the current user is in thenon-front-view state and then ends the video call.

As shown in FIG. 9, a device for a human-computer interaction in adisplay device includes:

an image data acquisition module 200 configured to acquire current imagedata of a user;

a display module 400 configured to display multiple different candidatedata in different display areas when detecting that the user is in afront-view state;

an recognition module 600 configured to identify a target display areawhere the user is gazing:

an execution module 800 configured to read candidate data correspondingto the target display area and perform an operation corresponding to theread candidate data.

In the above-mentioned device for the human-computer interaction in thedisplay device, the image data acquisition module 200 acquires thecurrent image data of the user. When detecting that the user is in thefront-view state, the display module 400 displays the multiple differentcandidate data in the different display areas. The user browses thecandidate data. The recognition module 600 recognizes the target displayarea that is being gazed. The execution module 800 reads the candidatedata corresponding to the target display area, and performs theoperation corresponding to the read candidate data. In the wholeprocess, user's needs can be perceived without excessive operation,which realizes a convenient human-computer interaction, and bringsconvenient operation to users.

In one of the embodiments, the above-mentioned device for thehuman-computer interaction in the display device further includes:

an interrupt module configured to interrupt the operation correspondingto the read candidate data when detecting that the user is not in thefront-view state.

In one of the embodiments, the display module 400 is further configuredto, when detecting that the user is in the front-view state, count timefor which the user is in the front-view state; and when the time forwhich the user is in the front-view state exceeds a preset timethreshold, display multiple different candidate data in the differentdisplay areas.

In one of the embodiments, the display module 400 is also configured to,according to the current image data of the user, adopt the front-viewrecognition technology, preliminarily detect whether the user is in thefront-view state; when the preliminary detection result is that the useris in the front-view state, acquire a face image of the user; perform arecognition processing and; according to the recognition processingresult, display the multiple different candidate data in the differentdisplay areas. The recognition processing includes at least one of aface recognition, an age recognition, an facial expression recognition,and a gender recognition.

In one of the embodiments, the display module 400 is further configuredto: when the preliminary detection result is that the user is in thefront-view state, perform the face recognition on the user; whenrecognizing that the user is a known legitimate user, acquire the faceimage of the user.

In one of the embodiments, the display module 400 is further configuredto, when recognizing the user's identity by the face recognition,display the multiple different candidate data associated with the user'sidentity in the different display areas.

In one of the embodiments, the recognition module 600 adopts a head poserecognition technology to confirm an orientation and an angle of theuser's head; recognizes a display area being browsed by the useraccording to the orientation and the angle of the user's head; and usesthe display area being browsed by the user as the candidate displayarea; cumulatively counts a staying time of the user's gaze in differentcandidate display areas, respectively; selects the candidate displayarea corresponding to the staying time that firstly and cumulativelyexceeds a preset gazing time threshold as the target display area wherethe user is gazing.

For the specific definition of the device for the human-computerinteraction in the display device, reference can be made to the abovedefinition of the method for the human-computer interaction in thedisplay device, which will not be repeated here. Each module in theabove-mentioned device for the human-computer interaction in the displaydevice can be implemented in whole or in part by software, hardware, anda combination thereof. The above-mentioned modules may be embedded in aprocessor in the computer device in a form of hardware or independent ofthe processor in the computer device, or may be stored in a memory ofthe computer device in a form of software, such that the processor cancall and perform the operations corresponding to each module asdescribed above.

In an embodiment, a computer device is provided. The computer device maybe a server. An internal structure diagram of the computer device may beas shown in FIG. 10. The computer device includes a processor, a memory,a network interface and a database connected through a system bus. Theprocessor of the computer device is configured to provide calculationand control capabilities. The memory of the computer device includes anon-transitory storage medium and an internal memory. The non-transitorystorage medium stores an operating system, a computer program, and adatabase. The internal memory provides an environment for the operationof the operating system and computer programs in the non-transitorystorage medium. The database of the computer device is configured tostore the image data of the user in the front-view state. The networkinterface of the computer device is configured to communicate with anexternal terminal through a network connection. The computer program isexecuted by the processor to implement a method for the human-computerinteraction in a display device.

Those skilled in the art can understand that the structure shown in FIG.10 is only a block diagram of part of the structure related to thesolution of the present application, and does not constitute alimitation on the computer device to which the solution of the presentapplication is applied. The specific computer device may include more orless components than shown in the figure, or combine with somecomponents, or have a different arrangement of the components.

In an embodiment, a computer device is provided, which includes amemory, a processor, and a computer program stored in the memory andcapable of being run in the processor. When executing the computerprogram, the processor implements the following steps:

Current image data of a user is obtained;

When detecting that the user is in a front-view state, multipledifferent candidate data are displayed in different display areas;

A target display area where the user is gazing is recognized;

The candidate data corresponding to the target display area is read, andan operation corresponding to the read candidate data is performed.

In an embodiment, when executing the computer program, the processorfurther implements the following step:

When detecting that the user is not in the front-view state, theoperation corresponding to the read candidate data is interrupted.

In an embodiment, when executing the computer program, the processorfurther implements the following steps:

When detecting that the user is in the front-view state, the time forwhich the user is in the front-view state is counted. When the time forwhich the user is in the front-view state exceeds a preset timethreshold, the multiple different candidate data are displayed in thedifferent display areas.

In an embodiment, when executing the computer program, the processorfurther implements the following steps:

According to the current image data of the user, the front-viewrecognition technology is adopted to preliminarily detect whether theuser is in the front-view state. When the preliminary detection resultis that the user is in the front-view state, the face image of the useris acquired. According to the face image of the user, the recognitionprocessing is performed. According to the recognition processing result,the multiple different candidate data are displayed in the differentdisplay areas. The recognition processing includes at least one of aface recognition, an age recognition, an facial expression recognition,and a gender recognition.

In an embodiment, when executing the computer program, the processorfurther implements the following step:

When recognizing the user's identity by the face recognition, multipledifferent candidate data associated with the user's identity aredisplayed in the different display areas.

In an embodiment, when executing the computer program, the processorfurther implements the following steps:

When the preliminary detection result is that the user is in thefront-view state, the face recognition is performed on the user. Whenrecognizing that the user as a known legitimate user, the face image ofthe user is acquired.

In an embodiment, when executing the computer program, the processorfurther implements the following steps:

A head pose recognition technology is adopted to confirm an orientationand an angle of the user's head. According to the orientation and theangle of the user's head, the display area being browsed by the user isrecognized. The display area being browsed by the user is used as thecandidate display area. The staying time of the user gaze in thedifferent candidate display areas is cumulatively counted, respectively.The candidate display area corresponding to the staying time thatfirstly and cumulatively exceeds a preset gazing time threshold isselected as the target display area where the user is gazing.

In an embodiment, a computer-readable storage medium is provided, onwhich a computer program is stored. When being executed by a processor,the computer program implements the following steps:

Current image data of a user is acquired.

When detecting that the user is in a front-view state, multipledifferent candidate data are displayed in different display areas:

A target display area where the user is gazing is recognized;

Candidate data corresponding to the target display area is read, and anoperation corresponding to the read candidate data is performed.

In an embodiment, when being executed by the processor, the computerprogram further implements the following step:

When detecting that the user is not in the front-view state, theoperation corresponding to the read candidate data is interrupted.

In an embodiment, when being executed by the processor, the computerprogram further implements the following steps:

When detecting that the user is in the front-view state, the time forwhich the user is in the front-view state is counted. When the time forwhich the user is in the front-view state exceeds a preset timethreshold, the multiple different candidate data are displayed in thedifferent display areas.

In an embodiment, when being executed by the processor, the computerprogram further implements the following steps.

According to the current image data of the user, the front-viewrecognition technology is adopted to preliminarily detect whether theuser is in the front-view state. When the preliminary detection resultis that the user is in the front-view state, the face image of the useris acquired. According to the face image of the user, a recognitionprocessing is performed. According to the recognition processing result,the multiple different candidate data are displayed in the differentdisplay areas. The recognition processing includes at least one of aface recognition, an age recognition, an facial expression recognition,and a gender recognition.

In an embodiment, when being executed by the processor, the computerprogram further implements the following step:

When recognizing the user's identity by the face recognition, themultiple different candidate data associated with the user's identityare displayed in the different display areas.

In an embodiment, when being executed by the processor, the computerprogram further implements the following steps:

When the preliminary detection result is that the user is in thefront-view state, the face recognition is performed on the user. Whenrecognizing the user as a known legitimate user, the face image of theuser is acquired.

In an embodiment, when executing the computer program, the processorfurther implements the following steps:

A head pose recognition technology is adopted to confirm an orientationand an angle of the user's head. According to the orientation and theangle of the user's head, the display area being browsed by the user isrecognized. The display area being browsed by the user is used as thecandidate display area. The staying time of the user gaze in thedifferent candidate display areas is cumulatively counted, respectively.The candidate display area corresponding to the staying time thatfirstly and cumulatively exceeds a preset gazing time threshold isselected as the target display area where the user is gazing.

A person of ordinary skill in the art can understand that all or part ofthe processes in methods according to the aforementioned embodiments canbe implemented by instructing relevant hardware via a computer program,which can be stored in a non-transitory computer-readable storagemedium. When the computer program is executed, the processes of thevarious methods according to the aforementioned embodiments may beincluded. Where, any reference to memory, storage, database or othermedium used in the embodiments according to this application may includea non-transitory memory and/or a transitory memory. The non-transitorymemory may include a read-only memory (ROM), a programmable ROM (PROM),an electrically programmable ROM (EPROM), an electrically erasableprogrammable ROM (EEPROM), or flash memory. The transitory memory mayinclude a random-access memory (RAM) or an external cache memory. As anillustration but not a limitation, RAM is available in many forms, suchas static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM),double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchlinkDRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM(DRDRAM), memory bus dynamic RAM (RDRAM), and the like.

The technical features of the aforementioned embodiments can be combinedarbitrarily. In order to simply the description, all possiblecombinations of the technical features in the aforementioned embodimentsare not described. However, as long as there is no contradiction in thecombinations of these technical features, they should be considered tobe fallen into the range described in the present specification.

Only several implementations of the present application are illustratedin the aforementioned embodiments, and the description thereof isrelatively specific and detailed, but it should not be understood as alimitation on the scope of the present application. It should be notedthat for those of ordinary skill in the art, without departing from theconcept of the present application, several modifications andimprovements can be made, which all fall within the protection scope ofthe present application. Therefore, the protection scope of the presentapplication shall be subject to the appended claims.

What is claimed is:
 1. A method for a human-computer interaction in adisplay device, comprising: obtaining current image data of a user; whendetecting that the user is in a front-view state, displaying multipledifferent candidate data in different display areas; recognizing atarget display area where the user is gazing; and reading the candidatedata corresponding to the target display area, and performing anoperation corresponding to the read candidate data, wherein therecognizing the target display area where the user is gazing comprises:adopting a head pose recognition technology to confirm an orientationand an angle of the user's head; according to the orientation and theangle of the user's head, recognizing a display area being browsed bythe user, and using the display area being browsed by the user as thecandidate display area; respectively counting the staying time of theuser's gaze in the different candidate display areas; and selecting thecandidate display area corresponding to the staying time that firstlyand cumulatively exceeds a preset gazing time threshold as the targetdisplay area where the user is gazing.
 2. The method according to claim1, further comprising, after reading the candidate data corresponding tothe target display area, and performing the operation corresponding tothe read candidate data: when detecting that the user is not in thefront-view state, interrupting the operation corresponding to the readcandidate data.
 3. The method according to claim 1, wherein the whendetecting that the user is in the front-view state, displaying themultiple different candidate data in the different display areascomprises: when detecting that the user is in the front-view state,counting time for which the user is in the front-view state; and whenthe time for which the user is in the front-view state exceeds a presettime threshold, displaying the multiple different candidate data in thedifferent display areas.
 4. The method according to claim 1, wherein thewhen detecting that the user is in the front-view state, displaying themultiple different candidate data in the different display areascomprises: according to the current image data of the user, adopting afront-view recognition technology to preliminarily detect whether theuser is in the front-view state; when the preliminary detection resultis that the user is in the front-view state, acquiring a face image ofthe user; and according to the face image of the user, performing arecognition processing; according to the recognition processing result,displaying the multiple different candidate data in the differentdisplay areas, wherein the recognition processing comprises at least oneof a face recognition, an age recognition, a facial expressionrecognition, and a gender recognition.
 5. The method according to claim4, wherein the when the preliminary detection result is that the user isin the front-view state, acquiring the face image of the user comprises:when the preliminary detection result is that the user is in thefront-view state, performing the face recognition on the user; and whenrecognizing the user as a known legitimate user, acquiring the faceimage of the user.
 6. The method according to claim 5, furthercomprising: when recognizing the user's identity by the facerecognition, displaying multiple different candidate data associatedwith the user's identity in the different display areas.
 7. The methodaccording to claim 5, further comprising, after acquiring the face imageof the user: when a size of the face image of the user is greater than apreset face image size threshold, displaying the multiple differentcandidate data in the different display areas.
 8. A computer device,comprising a processor storing a computer program and a memory, whenexecuting the computer program, the processor implements the steps of:obtaining current image data of a user; when detecting that the user isin a front-view state, displaying multiple different candidate data indifferent display areas; recognizing a target display area where theuser is gazing; and reading the candidate data corresponding to thetarget display area, and performing an operation corresponding to theread candidate data, wherein the recognizing the target display areawhere the user is gazing comprises: adopting a head pose recognitiontechnology to confirm an orientation and an angle of the user's head;according to the orientation and the angle of the user's head,recognizing a display area being browsed by the user, and using thedisplay area being browsed by the user as the candidate display area;respectively counting the staying time of the user's gaze in thedifferent candidate display areas; and selecting the candidate displayarea corresponding to the staying time that firstly and cumulativelyexceeds a preset gazing time threshold as the target display area wherethe user is gazing.
 9. A non-transitory computer-readable storagemedium, on which a computer program is stored, when being executed by aprocessor, the computer program implements the steps of: obtainingcurrent image data of a user; when detecting that the user is in afront-view state, displaying multiple different candidate data indifferent display areas; recognizing a target display area where theuser is gazing; and reading the candidate data corresponding to thetarget display area, and performing an operation corresponding to theread candidate data, wherein the recognizing the target display areawhere the user is gazing comprises: adopting a head pose recognitiontechnology to confirm an orientation and an angle of the user's head;according to the orientation and the angle of the user's head,recognizing a display area being browsed by the user, and using thedisplay area being browsed by the user as the candidate display area;respectively counting the staying time of the user's gaze in thedifferent candidate display areas; and selecting the candidate displayarea corresponding to the staying time that firstly and cumulativelyexceeds a preset gazing time threshold as the target display area wherethe user is gazing.
 10. The computer device according to claim 8, theprocessor further executes the computer program to implement, afterreading the candidate data corresponding to the target display area, andperforming the operation corresponding to the read candidate data: whendetecting that the user is not in the front-view state, interrupting theoperation corresponding to the read candidate data.
 11. The computerdevice according to claim 8, wherein the when detecting that the user isin the front-view state, displaying the multiple different candidatedata in the different display areas comprises: when detecting that theuser is in the front-view state, counting time for which the user is inthe front-view state; and when the time for which the user is in thefront-view state exceeds a preset time threshold, displaying themultiple different candidate data in the different display areas. 12.The computer device according to claim 8, wherein the when detectingthat the user is in the front-view state, displaying the multipledifferent candidate data in the different display areas comprises:according to the current image data of the user, adopting a front-viewrecognition technology to preliminarily detect whether the user is inthe front-view state; when the preliminary detection result is that theuser is in the front-view state, acquiring a face image of the user; andaccording to the face image of the user, performing a recognitionprocessing; according to the recognition processing result, displayingthe multiple different candidate data in the different display areas,wherein the recognition processing comprises at least one of a facerecognition, an age recognition, a facial expression recognition, and agender recognition.
 13. The computer device according to claim 12,wherein the when the preliminary detection result is that the user is inthe front-view state, acquiring the face image of the user comprises:when the preliminary detection result is that the user is in thefront-view state, performing the face recognition on the user; and whenrecognizing the user as a known legitimate user, acquiring the faceimage of the user.
 14. The computer device according to claim 13, theprocessor further executes the computer program to implement: whenrecognizing the user's identity by the face recognition, displayingmultiple different candidate data associated with the user's identity inthe different display areas.
 15. The non-transitory storage mediumaccording to claim 9, the processor further executes the computerprogram to implement, after reading the candidate data corresponding tothe target display area, and performing the operation corresponding tothe read candidate data: when detecting that the user is not in thefront-view state, interrupting the operation corresponding to the readcandidate data.
 16. The non-transitory storage medium according to claim9, wherein the when detecting that the user is in the front-view state,displaying the multiple different candidate data in the differentdisplay areas comprises: when detecting that the user is in thefront-view state, counting time for which the user is in the front-viewstate; and when the time for which the user is in the front-view stateexceeds a preset time threshold, displaying the multiple differentcandidate data in the different display areas.
 17. The non-transitorystorage medium according to claim 9, wherein the when detecting that theuser is in the front-view state, displaying the multiple differentcandidate data in the different display areas comprises: according tothe current image data of the user, adopting a front-view recognitiontechnology to preliminarily detect whether the user is in the front-viewstate; when the preliminary detection result is that the user is in thefront-view state, acquiring a face image of the user; and according tothe face image of the user, performing a recognition processing;according to the recognition processing result, displaying the multipledifferent candidate data in the different display areas, wherein therecognition processing comprises at least one of a face recognition, anage recognition, a facial expression recognition, and a genderrecognition.
 18. The non-transitory storage medium according to claim17, wherein the when the preliminary detection result is that the useris in the front-view state, acquiring the face image of the usercomprises: when the preliminary detection result is that the user is inthe front-view state, performing the face recognition on the user; andwhen recognizing the user as a known legitimate user, acquiring the faceimage of the user.
 19. The non-transitory storage medium according toclaim 18, the processor further executes the computer program toimplement: when recognizing the user's identity by the face recognition,displaying multiple different candidate data associated with the user'sidentity in the different display areas.